Combined Use of Reinforcement Learning and Simulated Annealing: Algorithms and Applications

] , [ b a , ] , [ d c real intervals w ∇ partial derivatives of a quantity according to the weight vector elements i A , i B , i C machining times on machine s A, B, and C respectively j i a jth action in the ith state of the agent α learning rate for " mean value type " action-state values b vector of boundary conditions in flow-shop scheduling β learning rate for " standard deviation type " action-state values β " inverse temperature " (Chapter 3) C set of constraints (Chapter 5) C number of precedence constraints c , i c constant parameters (Chapter 3) i c precedence constraints (Chapter 5) c vector of parameters (Chapter 3) ij c cost of delivery between nodes i and j (Chapter 4) γ discount factor D compound matrix i d distance label on node i ij d distance label on node i (matrix representation)) dim(a dimension of vector a δ temporal difference factor } {x E π expected value of x following policy π i E " potential energy ") (s e t eligibility value of state s e the natural number ... 71. 2 ≈ i e base vector in dimension i (Chapter 5) ε small positive error) (x f transfer function) (' x f ,) (" x f reward function) (x g reward function g auxiliary machining-delay vector (Chapter 5)) (' x h ,) (" x h reward function 8 j I Palmer index for job j ij η visibility of a trail from node i to node j j j the jth job under processing k the number of actions sharing the same, maximal action-state value κ minimum of the difference between each action-state value pair) (A L link state description of node A λ decay rate for eligibility traces M matrix of machining times) (i m number of actions in the ith state (Chapter 2) n number of states ν small positive redundancy variable 1 o , 2 o abstract machining times a ss P ' probability of getting into state ' s from state s via action a i p probability of selecting action i a p ˆ scaled probability value) (x p continuous probability distribution function p , r permutation vectors (Chapter 5) i pred predecessor node ij pred predecessor node (matrix representation)) , (j i i a s π …